Supporting learners: Increasing complexity?

The different papers in this special issue all addressed learning with complex tasks. All of these papers reached only partially the expected results. This discussion on possible factors that may explain these unexpected results. A first issue that is questioned is the functionality of the tools in the studies. Secondly, the learners lack of compliance is addressed. It may have been that the learners did not take the opportunities offered to them. Third, the use of the support by the learners itself is questioned. Although some methodological issues can be raised, the different papers made a worthwhile attempt to grasp the complexity in a learning environment. Moreover, they highlight the importance of a consolidated framework to determine relevant factors that should be considered when dealing with complexity. 2006 Elsevier Ltd. All rights reserved. At the occasion of the second joint meeting of the EARLI special interest groups ‘Instructional Design’ and ‘Learning and Instruction with computers’ six papers were brought together for this special issue. Each paper deals with a learning environment that offers a complex learning task to foster the development of higher cognitive skills, such as connecting information found on the internet (Narciss, Proske, & Koerndle), integrating multiple representations (Seufert, Jänen, & Brünken), building an argumentation (Janssen, Erkens, & Kanselaar; Munneke, Andriessen, Kanselaar, & Kirschner), performing a task analysis (Van Berlo, Lowyck, & Schaafstal), or building mental models (Huk & Steinke). All these studies nicely illustrate that learners have difficulties in coping with these complex tasks, and the studies assume that executing the tasks will result in increased performance, better understanding, better reasoning, and ultimately better learning. 0747-5632/$ see front matter 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.chb.2006.10.008 * Corresponding author. Tel.: +32 16 325733. E-mail address: [email protected] (J. Elen). J. Elen, G. Clarebout / Computers in Human Behavior 23 (2007) 1162–1166 1163 Each of the studies reports on a specific intervention to deal with this problem. The papers of Seufert et al. and of Munneke et al., for example, used a graphic organizer to support learners; Huk and Steinke and Janssen et al. used visualization techniques, and Narciss et al. and Van Berlo et al. provided support for the learning process itself. Three of the papers (Seufert et al.; Huk & Steinke; Narciss et al.) addressed this issue in an individual setting. The three other contributions (Janssen et al.; Munneke et al.; Van Berlo et al.) addressed complex learning in a collaborative setting with two of them using the computer as the means for communication. Despite the adequate design of the interventions and the studies, in most of the studies the expected results are only partially reached. The graphic organizers in the Seufert et al. study seemed not to have been beneficial. The hypothesis in the study of Munneke et al. that students working with an argumentation diagram would perform better than students with an outline tool was not confirmed. The visualization techniques in the Huk and Steinke study led to opposite results than those hypothesized. Students with a high spatial ability benefited more from the connecting lines tool than did students with a low spatial ability, while this tool supplants an activity which low spatial ability students most likely cannot perform themselves. And, while Janssen et al. are quite positive about their results, their results with respect to collaborative activities and group norms are only marginally significant. It remains a question whether the reported positive perceptions of students are due to a novelty effect. With respect to the learning process-support, the monitoring tool was poorly used in the Narciss et al. study, and in the Van Berlo et al. study, the scenarios and workshop did not lead to a significant increase in the quality of the task analysis and the design process. Rather than neglecting the ‘unexpected results’, they trigger reflections on possible factors that may have generated them. These factors relate to the instructional intervention itself, the learner, some methodological aspects, and the interaction between these variables. First of all, and as an essential prerequisite, we need to consider whether the instructional interventions are actually functional (Elen & Clarebout, 2006). Building on previous studies or theoretical reasoning, the functionality of the tools and guidelines in the studies are largely taken for granted. The tools and guidelines are considered to be helpful and functional for all learners to carry out a particular task, or solve a particular problem. Nevertheless, the functionality of at least some of the tools can be questioned. In both the Janssen et al. and Munneke et al. studies students used computers to collaborate and communicate while working in each other physical presence. As Munneke et al. state: ‘‘students worked on their own computer, in most cases without being able to see each other’’. In the Janssen et al. study, students collaborated and were expected to use a shared space to visualize the discussion while working in the same room. While there might have been serious research reasons to put students in the same room (the authors point to the need for ecological validity), individually using computers to create a virtual social environment while staying in the same room hardly can be called functional or helpful. Why would anybody use the computer to communicate when working in the same room? The specific research setting may have decreased the functionality of the tools. While the authors try to increase the ecological validity of their study, they rather decrease it by creating a setting in which the support provided is no longer functional. A second reason for the unexpected, suboptimal use of the tools may relate to the learner’s lack of compliance. Perkins (1985) indicated that the learner has to take the 1164 J. Elen, G. Clarebout / Computers in Human Behavior 23 (2007) 1162–1166 opportunities offered in a learning environment in order for these opportunities to be effective. The learner has to be willing to engage in meaningful learning activities (Winne, 1987). In the Seufert et al. article, it is indeed noted that a learner has to be able and willing to invest mental effort in the task. So in addition to functionality, a motivational issue also needs attention. The Narciss et al. paper clearly indicates that the participants did not comply with the environment, or that they did not take the opportunities offered, since only a small amount of tool use was reported. Similarly, in the Munneke et al. study the lack of compliance could be an explanation of the unexpected results. In addition to the functionality of the support for learning to solve complex problems and the compliance of the learners, a third question pertains to the use of the support by the learners. Did the learners use the support provided and did they use it as intended? The studies in this issue build on the assumption that by adding tools or by providing guidelines to learners, learners get actual guidance for their learning process, and hence learners’ performance will increase. Learners are expected to know how to use the support for improving their learning. This assumption cannot be taken for granted as the tools and guidelines introduce additional complexity or even an additional task (de Jong, de Hoog, & de Vries, 1992). Students have to learn to work with the tools and grasp the functionality of the tool for the actual learning task. Therefore, tools may only help learners to carry out tasks when they clearly see how these tools can help them. In all other cases, tools increase complexity by inserting new elements whose relevance has to be determined by the learner as part of carrying out the task. In cognitive load terms: while in most studies the aim is to decrease extraneous cognitive load, adding support may actually increase the total cognitive load. Strikingly, cognitive load gets addressed in different papers (Seufert et al.; Munneke et al.; Huk & Steinke). The different papers reveal the complexity of the notion and the methodological difficulties to actually consider it in an empirical work. A first – methodological – issue relates to the assessment of cognitive load and more specifically of ‘intrinsic cognitive load’. Seufert et al. focus on the impact of intrinsic cognitive load on tool use. Intrinsic load is argued to consist of externally determined intrinsic cognitive load and internally determined intrinsic load. The authors claim that they can keep the internally determined intrinsic load constant by taking a random sample for their experiment. That might be the case, but it requires that two conditions are met; first, the variable on which the sample is randomized needs to be specified (e.g., intrinsic load, prior knowledge, etc.), and second, the variable needs to be assessed. A second issue relates to the relationship between cognitive load induced by the learning task and the possible additional load induced by the support. For instance, in the Seufert et al. study, learner control on the use of visual hints may have added cognitive load to the one induced by task complexity. A third issue pertains to the over-use of cognitive load as an explanation without considering alternatives. While cognitive load theory may, in certain cases, present a well-suited interpretation framework, it does not always rule out alternatives. In any case, if cognitive load is considered it also needs to be studied. For instance, the Seufert et al. study is claims that tool use is affected by task complexity. The authors argue that the poor use of hyperlinks is an indication that learners experienced a high degree of cognitive load and that they avoided an additional overload. This might be a possible explanation, but an alternative explanation, as already previously indicated, could be that students did not see the benefit of the hyperlinks and hence decided not to use them. In other words, they did J. Elen, G. Clarebout / Computers in Human Behavior 23 (2007) 1162–1166 1165 not feel overloaded but they had no idea about the functionality of the tool. Munneke et al. argue that cognitive load may be one of the reasons why people have difficulty with deep argumentation. If this is a probable explanation, it may need to be explicitly considered in designing a following study. The same applies to the Huk and Steinke study. Overall, it can be said that despite the cognitive orientation of the papers, learners with their specific characteristics are only marginally addressed. For example, the possibility that students perceive the functionality of the support differently from the intention of the designer (Winne, 1985) is only considered by Van Berlo et al. as a possible explanation of their results. A designer assumes a certain functionality of the support, but this designed functionality does not necessarily correspond to the perceived functionality. A clear explanation of, and training in the use of the tool could have increased the effect on collaborative activities (Janssen et al.), or engagement in argumentative reasoning (Munneke et al.). Similarly, the positive effects of training in the use of the guidelines could have been expected in the Van Berlo et al. studies. In addition to support-perception, task-perception may also affect the effect of instructional interventions. When students perceive a learning task as a performance task, aimed at reaching a product as quickly as possible (Munneke et al.), the tools are merely a burden if they do not help to achieve the goal. Rather than enhancing certain cognitive skills, the support forms an additional source of extraneous cognitive load. Additionally, not only the perception of the kind of task may influence the beneficial effect of support; also the perceived task complexity may co-determine how much mental effort learners invest in the task (Clark, Howard, & Early, 2006). These aspects indicate that it is important to consider the learner as well when designing and implementing interventions. In the different papers, the learner is only considered in a limited way. In the Huk and Steinke study spatial ability is considered as an important learner characteristic. Narciss et al. on the other hand, consider metacognition as an important variable for determining the success of learners when dealing with a complex task, but they do only consider it as a dependent variable, assuming that their group of participants are on an equal level with respect to their metacognition, since they are second year students. Overall, the different contributions in this special issue lack a clear theoretical model that identifies the important variables to be considered in the empirical studies. One paper even does a post-test only study (Janssen et al.), assuming that learner characteristics do not play a role. It looks like a positive bias towards the effect of instructional interventions, as if the mediating role of the learner can be ignored. If one would assume that learner characteristics are not important to consider, or that one can assume that the support offered is beneficial for all students, the question can be raised why one would leave the learners in control of the use of the support (Elen & Clarebout, 2006). If the support is beneficial for all students, one should design it as such that the learner cannot but use it. Moreover, if the support is well designed, the use itself cannot but be adequate. A last aspect, already partially addressed, is a methodological concern. Ecological validity is claimed by different authors because they perform their study in a real classroom setting. However, the kind of environment offered in the study does not correspond to a regular classroom environment, learners are confronted with a task does not correspond to a regular class setting. The problem with the ecological validity of the study of Janssen et al. and Munneke et al. was already mentioned, but also Huk and Steinke present a study in which they claims to increase ecological validity by bringing the 1166 J. Elen, G. Clarebout / Computers in Human Behavior 23 (2007) 1162–1166 computers into a regular class. When a regular classroom setting is completely adapted to a research study with a 20-min intervention, the ecological validity of this study is similar or even worse than in an ‘ordinary’ experiment. Additionally different papers want to assess cognitive load, and struggle with it. This may be because of the interactive nature of this variable. It is not a pure learner variable or a pure task-related variable, it is the results of an interaction between learner and task characteristics. As such it is hard to control and assess this variable in research. The different studies all made a worthwhile attempt to address the issue of complexity in a learning environment. Each of these papers tried to gain an insight into the issue of support. Moreover, they do also highlight the importance of a clear consolidated theoretical framework to determine relevant factors that should be considered when dealing with complex learning tasks. This theoretical model can help in designing support that can enhance complex problem solving. But before studying the effect of these tools on learning, one needs to study the quality of the tools themselves.